Running Head: Improving Chest X-Ray Analysis Using Eye Movement Cues Viewing Another Person’s Eye Movements Improves Identification of Pulmonary Nodules in Chest X-Ray Inspection

Double reading of chest x-rays is often used to ensure that fewer abnormalities are missed, but very little is known about how the search behavior of others affects observer performance. A series of experiments investigated whether radiographers benefit from knowing where another person looked for pulmonary nodules, and whether the expertise of the model providing the search behavior was a contributing factor. Experiment 1 compared the diagnostic performance of novice and experienced radiographers examining chest x-rays and found that both groups performed better when shown the search behavior of either a novice radiographer or an expert radiologist. Experiment 2 established that benefits in performance only arose when the eye movements shown were related to the search for nodules; however, only the novices’ diagnostic performance consistently improved when shown the expert’s search behavior. Experiment 3 re-examined the contribution of task, image, and the expertise of the model underlying this benefit. Consistent with Experiment 1, novice radiographers were better at identifying nodules when shown either a naïve’s search behavior or an expert radiologist’s search behavior, but they demonstrated no improvement when shown a naïve model not searching for nodules. Our results suggest that although the benefits of this form of attentional guidance may be short-lived, novices can scaffold their decisions based on the search behavior of others. Viewing Another Person’s Eye Movements 3 Viewing Another Person’s Eye Movements Improves Identification of Pulmonary Nodules in Chest X-Ray Inspection Around 40% of all hospital procedures require some form of medical image interpretation (Beam, Krupinski, Kundel, Sickles, & Wagner, 2006). For successful diagnosis it is essential that those who analyze these images are able to interpret correctly the information that they contain. When examining chest x-rays one of the many features that radiologists look for is the presence of pulmonary nodules because these can be precursors to lung cancer. The radiologist must correctly distinguish these subtle abnormalities from normal features. However, the complexity of medical images of lungs can lead to relatively high numbers of false positive decisions (thinking that a feature is a nodule when, in fact, it is not) and false negative decisions or “misses” (thinking that a feature is normal when, in reality, it is a nodule; see Fitzgerald, 2001; Taylor, 2007). Previous work on false negative errors suggests that at least 30% arise from insufficient visual search (Kundel, Nodine, & Carmody, 1978; Manning, Ethell, & Donovan, 2004), with novice radiographers being particularly prone to making such search-related errors. With experience, however, decision-making and interpretation processes become the source of most errors (Donovan & Manning, 2006). Decisionmaking and interpretation skills require appropriate training and extensive exposure to case studies, which makes such skills difficult to develop rapidly. In contrast, viewing the eye movement behavior of another observer completing the same task may help with the development of visual scanning and recognition skills, as this could provide an example of an effective search process (e.g., following where another person looks may reveal to an observer the most task-relevant areas in an image such as probable nodule sites). It could be argued that experts would be better than novices at guiding observers’ attention to relevant task-specific areas, because experts would be more likely to locate Viewing Another Person’s Eye Movements 4 these areas in the first place (e.g., Kundel & Nodine, 1983; Nodine, Mello-Thoms, Kundel, & Weinstein, 2002). However, it is not simply the case that when observers fixate at nodule locations the correct decision is made. Indeed, the more controversial implication of Kundel et al.’s (1978) findings concerning false negative errors is that around 70% of nodules that are missed are in fact fixated (a finding replicated in domains such as baggage screening; McCarley, Kramer, Wickens, Vidoni, & Boot, 2004). As such, these nodule locations would have been given some degree of processing but were deemed to be normal. Making observers re-evaluate their decisions based on these perceptual factors is one way to reduce errors. For example, when radiologists are shown their own eye movement behavior during pulmonary nodule identification, this can lead to an improvement in performance (Krupinski, Nodine, & Kundel, 1993; Kundel, Nodine, & Krupinski, 1990). However, recent evidence using a similar paradigm shows that this perceptual feedback is mostly beneficial for radiographers in the early stages of learning (Donovan, Manning, & Crawford, 2008). The present study aims to expand our understanding of the factors that may improve pulmonary nodule identification by investigating whether viewing another person’s eye movements improves visual search and decision-making during a nodule detection task. The Communicative Role of Eye Movements The eye movements of experts and novices reveal how visual search strategies can change as a function of expertise (Chapman & Underwood, 1998; Rayner, 1998), however, eye movements can also be communicative and direct the attention of others (Kleinke, 1986). Studies have shown that we are highly sensitive to other people’s gaze (Gibson & Pick, 1963; Symons, Lee, Cedrone, & Nishimura, 2004) and that this sensitivity develops early, with very young infants demonstrating gaze following behavior (e.g., Brooks & Meltzoff, 2005; Corkum & Moore, 1995). Following another’s Viewing Another Person’s Eye Movements 5 gaze can guide attention towards a particular item, which allows for knowledge to be inferred or directly shared if the observers verbalise their interest and intent towards the jointly attended item (Flom & Pick, 2007; Hanna & Brennan, 2007). This “joint attention” has been claimed to underpin much of early learning (Butterworth & Jarret, 1991), but is still highly prevalent in adulthood (Driver et al., 1999; though see Vecera & Rizzo, 2006, for discussion of voluntary aspects of gaze orienting). In particular, adult observers take into account nearby objects when following another’s gaze, with recent neuropsychological evidence indicating that following a person’s gaze can transfer that individual’s intentionality to the observer (Becchio, Bertone, & Castiello, 2008; Lobmaier, Fischer, & Schwaninger, 2006). Thus by watching where another person looks we alter our processing of objects based on the processing of others. If a person’s eye movements can direct our attention and we can also learn to make use of these cues wherever possible, then how much of cognition is shaped this way? Eye movement behavior is often understood as a reflection of underlying cognitive activity (Vivianni, 1990), but recent research suggests that specific eye movement behaviors can also influence cognition (Grant & Spivey, 2003; Pomplun, Ritter, & Velichkovsky, 1996; Thomas & Lleras, 2007). Given the tendency for people to use another’s gaze to drive their own reasoning and decision-making, there is currently much interest in presenting the eye movement patterns of one observer to subsequent observers to explore how this form of attentional guidance improves performance in a range of visual search and problem solving tasks. In particular, because experts are known to look at task-relevant areas more often than novices and demonstrate effective search strategies (e.g., Chapman & Underwood, 1998; Charness, Reingold, Pomplun, & Stampe, 2001; Krupinski, 1996), there is a growing belief that an expert’s eye movement patterns should be particularly useful in training novices where to look (Krupinski et al., 2006; Viewing Another Person’s Eye Movements 6 Nalanagula, Greenstein, & Gramopadhye, 2006; Sadasivian, Greenstein, Gramopadhye, & Duchowski, 2005). By recording the eye movement behavior of experts and showing the projected fixation position to other observers, recent studies have demonstrated that novices can detect more faults during aircraft inspection (Sadasivian et al., 2005) and circuitry board inspection (Nalanagula et al., 2006). For tasks such as program debugging and puzzle solving, which involve factors in addition to visual search, viewing another’s gaze can result in shorter task completion times (Stein & Brennan, 2004; Velichkovsky, 1995). However, none of these studies systematically examined the role of either model expertise or participant expertise on task performance, such that it remains unclear whether a novice’s eye movements could also act as useful attentional cues. In addition, there are some situations where unambiguous verbal comments are preferred to another’s gaze (e.g., Bard et al., 2007; Van Gog, Jarodzka, Scheiter, Gerjets, & Paas, 2009). As such, positive effects will likely depend on the task demands and the different ways in which eye movement patterns can be presented (cf. Nalanagula et al., 2006). A final issue is that projecting gaze behavior to observers involves “artificially” represented gaze (eye-movement patterns dynamically overlaid on a screen-based image of a task), and therefore it is debatable whether the cognitive processes are the same as those during normal gaze perception and gaze following behavior. For example, the emotional expression of the person being watched is often taken into account when using normal gaze as a predictive cue (Bayliss, Frischen, Fenske, & Tipper, 2007) or when determining mutual gaze (Lobmaier, Tiddeman, & Perret, 2008). Although this information is absent when using artificial gaze, this method does provide an opportunity to look at more complex gaze sequences rather than simple directional processes. Indeed, there have been reports of higher order search strategies based on another’s real-time Viewing Another Person’s Eye Movements 7 gaze. For example, observers can regulate their own search behavior in a collaborative visual search task by strategically ignoring areas that they can see are being observed by their collaborator (Brennan, Chen, Dickinson, Neider, & Zelinsky, 2008). By taking advantage of these non-verbal gaze cues, observers have been shown to be able to reduce their search times significantly. Thus, even with artificially represented gaze, observers alter their information processing behavior based on the processing of others. Experiment 1 The evidence described above suggests that another’s gaze is often used to help direct attention and facilitate learning. Experiment 1 aimed to examine whether viewing another’s search behavior would improve an observer’s ability to identify pulmonary nodules. However, training requirements will differ depending on an observer’s expertise, such that an observer’s experience should be accounted for when adopting new instructional techniques. It is expected that showing eye movement patterns of an expert to novice radiographers will lead to improved decision-making performance. Conversely, showing an expert’s eye movement patterns to experienced radiographers may actually disrupt diagnostic performance (e.g., Donovan et al., 2008; Donovan, Manning, Philips, Higham, & Crawford, 2005), a possibility that links to the “expertise reversal effect” (Kalyuga, 2007), whereby instructional techniques designed to reduce cognitive load may inhibit the application of pre-existing skills. We designed Experiment 1 to examine these latter predictions relating to observer expertise. We also investigated whether model expertise had a modulating influence so as to test the assumption that viewing an expert model leads to greater improvement compared to viewing a novice model. The experiment involved three main conditions: (1) “free search”, where observers could immediately make decisions regarding nodules; (2) “image preview”, where observers were required to wait for 20s before making a decision; and (3) “eye Viewing Another Person’s Eye Movements 8 movement preview”, where observers viewed either an expert’s or a novice’s eye movements for 20s before making a decision, with half shown a novice’s and half shown an expert’s eye movements. The free search condition provided the closest approximation to natural performance during nodule detection, whereas the image preview condition controlled for the additional image exposure in the eye movement preview condition. Method Participants. The study involved 24 novice radiographers (9 male; 15 female) and 24 experienced radiographers (8 male; 16 female). Participants had normal or correctedto-normal vision and were paid £6. Novices were all recruited in the final half of their first year radiography degree, which allowed them to accrue several months of formal instruction, and 12 weeks of clinical experience. Hence, novices had at least some experience of medical image perception. Experienced radiographers were enrolled on postgraduate radiography courses and had at least 5 years experience interpreting medical images from a range of modalities, such as ultrasound, computed-tomography, and MRI. The experienced radiographers were expected to be more proficient than novice radiographers in this task, but less skilled than experts who deal primarily with chest xray interpretation. Materials. Seven normal and seven abnormal images were selected for each viewing condition from a test bank of 120 posteroanterior chest x-rays. Each viewing condition contained a total of 16 nodules: 1 image with 5 nodules, 2 images with 3 nodules, 1 image with 2 nodules, and 3 images with 1 nodule. Nodules ranged in size from approximately 0.5° to 3°. Nodules were controlled across conditions for size, shape and conspicuity and were verified by a consultant radiologist. Images were standardized to fit an LCD display (resolution of 1280 x 1024) and were presented using ClearView 2.6. (Tobii Technology, Stockholm, Sweden). Image order was randomised. Viewing Another Person’s Eye Movements 9 Design. The experiment employed a 2 x 2 x 3 mixed design with three factors: participant expertise with two levels (novice vs. experienced), model expertise with two levels (novice vs. expert), and viewing condition with three levels (free search; image preview; eye movement preview). The presentation of the three viewing conditions was completely counterbalanced. In the free search condition participants could make decisions regarding nodules immediately. In the image preview condition participants viewed each x-ray for 20s before being allowed to make a decision. The eye movement preview condition was identical to the image preview condition except that during the 20s viewing period participants were also shown an overlay of the dynamic eye movements of a pre-selected scanpath (see Nalanagula et al., 2006, for evidence that dynamic and hybrid gaze cues produce enhanced benefits in a visual-search training context when compared with static cues). For each presented scanpath the gaze cursor increased in size in accord with fixation duration. At any one time 500ms of fixation activity was shown, with transitions between fixations depicted by lines (see Figure 1). Participants were told they would be “shown an eye movement recording of someone else examining the image” and that after the recording finished they would be “presented with the same image and asked to search and identify nodules.” The only description of the eye movement recordings given to participants was that “the circle represents where someone looked, and the bigger the circle, the longer the individual looked at that particular area.” No information was provided regarding the expertise of the model, nor whether the model had provided a correct diagnosis. << Insert Figure 1 about here >> To obtain the model scanpaths, five radiologists (experts) and five first-year radiographers (novices) were paid to complete the nodule detection task under free search conditions using the film test bank described above. Their eye movements were recorded Viewing Another Person’s Eye Movements 10 using a Tobii x50 eye tracker (Tobii Technology, Stockholm, Sweden), a standalone remote eye-tracking device with an accuracy of 0.5 degrees and a sampling rate of 50Hz. The minimum fixation duration was set to 100ms with a fixation radius of 50 pixels. We used the best performing expert and the worst performing novice as the models to show to the participants in the main study. To assess the overall performance of each of these five experts and five novices (and the performance of observers in the main study), a typical approach would be to use signal detection theory, which involves fitting a standard receiver operating characteristic (ROC) curve based on whether a series of images are correctly identified as being abnormal or normal (Metz, 2008). This method overcomes the problem that scoring the percentage of correct decisions neither reflects decision uncertainty nor the relatively high number of false positives that arise in medical image perception. However, a fundamental limitation of ROC methodology is that it is only suitable for binary decisions (e.g., normal/abnormal) and cannot account for the multiple decisions and/or abnormalities that can occur on the same image (Chakraborty, 2000). Moreover, the requirements of ROC allow an observer to decide an image is abnormal without actually specifying the location of any abnormalities. Based on these limitations, the present study adopted a free-response (FROC) method that required observers to locate and identify any nodules that were present, or decide that the image was normal. Currently, the most appropriate method of analysing FROC data is to perform a jackknife free-response ROC (JAFROC) analysis as this has shown greater statistical power than ROC and “alternative FROC” (AFROC) analysis (Chakraborty, 2006; Chakraborty & Berbaum, 2004). By applying the non-parametric Mann-Whitney-Wilcoxon U-statistic to the nodule ratings and the ROC-equivalent ratings of normal images, a “figure-of-merit” is estimated, which represents the likelihood that a true positive will be given a higher rating than a false positive. This Viewing Another Person’s Eye Movements 11 single measure of observer performance takes into account continuous ratings of confidence and reflects the error in decision-making in a FROC designed study. With .50 being at chance level, a high figure-of-merit (e.g., above .75) would mean that the majority of abnormalities were confidently identified, with few detrimental false positive decisions. However, a low figure-of-merit (e.g., below .50) would mean that even though some abnormalities may have been identified, performance would have been offset by a substantial number of detrimental false positive decisions. Accordingly, the eye movement patterns from the novice with the lowest figure-of-merit score (.66) and the radiologist with the highest figure-of-merit score (.79) were selected as the model scanpaths for the eye movement preview condition. Half the participants in the eye movement preview condition were shown the expert model and half the novice model. Procedure. Participants were seated approximately 60cm from the display and were asked to search for nodules from the test bank described previously. Five practice images were shown before the experiment, demonstrating how nodules differed in size, shape and conspicuity. When participants believed they had identified a nodule they used the mouse to indicate its location and clicked the left mouse button to confirm their decision. Once the decision was made, participants rated how confident they were by calling out a number between 1 and 4, with 4 being “highly confident.” Participants pressed a key when they thought they had identified all the nodules on an image, or if they thought an image contained no nodules. Participants were not told how many nodules could be present on each image and were unaware of the prevalence of nodules (50%) across images. The whole experiment took approximately 1 hour to complete. Results and Discussion Ratings of true positives and false positives were subjected to a JAFROC analysis as described above (see Chakraborty & Berbaum, 2004). The figure-of-merit scores Viewing Another Person’s Eye Movements 12 obtained from the JAFROC analysis were then examined using a mixed design ANOVA, where the between factors were participant expertise (novice vs. experienced) and model expertise (novice vs. expert), and where the within factor was viewing condition (free search; image preview; eye movement preview). This analysis revealed a main effect of viewing condition [F(2, 88) = 30.47, p < .001, partial η = .41], with performance improvements being evident for both novice and experienced radiographers in eye movement preview compared to image preview and free search (see Table 1). We note, however, that the magnitude of the difference was relatively small. Participant expertise was not significant [F(1, 44) = 2.00, p > .16, partial η = .04], however, a reliable interaction was found between participant expertise and viewing condition [F(2, 88) = 4.29, p < .017, partial η = .09]. The main effect of model expertise was not significant [F(1, 44) = 3.16, p > .08, partial η = .07], and model expertise did not interact with any other factors [all Fs < 2.19, all ps > .12]. These latter findings suggest that the level of expertise of the model that provided the eye movement recordings was not an influential factor in Experiment 1. << Insert Table 1 about here >> To explore the significant interaction between participant expertise and viewing condition we conducted a series of simple main effects analyses. Both novice and experienced radiographers showed a reliable performance difference across conditions [F(2, 46) = 25.24, p < .001, partial η = .52 for novices; F(2, 46) = 6.84, p < .002, partial η = .23, for experienced]. Bonferroni-corrected pairwise comparisons confirmed that for both novices and experienced radiographers the eye movement preview condition led to higher scores than the free search and image preview conditions (all ps < .05). Simple main effects comparing across participant expertise levels at each condition only revealed a reliable difference for image preview [F(1, 93.95) = 6.24, p < .014]. These analyses Viewing Another Person’s Eye Movements 13 clarify that both groups benefit from receiving eye movement patterns before making decisions regarding nodules when compared with free search or image preview. The analyses also indicate that the cause of the interaction between participant expertise and viewing condition relates to the performance separation between experienced and novice radiographers in the image preview condition. The overall decision times for each condition (see Table 1) were analyzed to examine if eye movement preview increased decision times, as seen in studies of aircraft inspection (Sadasivian et al., 2005). An equivalent mixed design ANOVA was employed as in the analysis of JAFROC scores. Decision time was measured from the time the image was presented until the participant terminated the search with a key press. A main effect of viewing condition was found [F(2, 88) = 69.35, p < .001, partial η = .61], with Bonferroni-corrected pairwise comparisons indicating significantly longer decision times for image preview relative to free search (p < .001) and longer decision times for eye movement preview in comparison to both image preview and free search (both ps < .001). The important comparison across these conditions is between the image preview and eye movement preview conditions since these both required participants to view each image for a minimum of 20s as distinct from the control condition where this 20s viewing requirement did not arise. No difference in decision time was found for participant expertise [F(1, 44) = 1.36, p > .20, partial η = .03] and there was no interaction between participant expertise and viewing condition [F(2, 88) = .06, p > .20, partial η < .01]. Model expertise showed neither a main effect nor any interactive effects with other factor [all Fs < 1.48, all ps > .23]. Overall our results indicate that both groups of radiographers demonstrated significantly improved performance when shown another person’s eye movements. The improvement by novices was expected because their underdeveloped visual search Viewing Another Person’s Eye Movements 14 behavior has been linked to their relatively poor performance in x-ray image analysis (Kundel et al., 1978). What was not expected was that experienced radiographers would also benefit from being shown another person’s eye movements. It was predicted that experienced radiographers would have already encountered enough cases to have developed effective scanning and search strategies, such that knowing where another person looked would only distract them from implementing these strategies. The lack of a performance difference in the eye movement conditions may indicate that: (1) insufficient cases were presented to enable experience related differences to emerge; or (2) that the experienced radiographers had not encountered enough chest x-ray images in their professional practice to develop the aforementioned search strategies. Thus, although experienced radiographers were generally more proficient than novice radiographers, any declining utility of gaze following may have been more apparent with expert radiologists specialized in chest x-ray interpretation. Another finding relating to observer performance concerned the interaction between participant expertise and viewing condition, whereby novice performance was worse in the image preview condition compared with that of experienced radiographers. This effect may have arisen as a negative consequence of increasing the time available for the novices to examine presented images in the absence of any useful visual cues. This interpretation is consistent with previous studies showing that extended viewing of x-rays often leads to an increase in the number of false positives (Christensen et al., 1981; Edwards, Ricketts, Dubbins, Roobottom, & Wells, 2003; Manning et al., 2004), whereas faster decision times are typically an indicator of expertise (Nodine et al., 2002). In the presence of useful visual cues, however, it may well be that longer decision times are associated with performance gains. This is precisely what was found in the eye movement preview condition, where both groups of radiographers took longer to perform Viewing Another Person’s Eye Movements 15 the task but also had the highest figure-of-merit scores. We propose that more than just visual search is involved in the eye movement preview condition in our study. For example, in addition to the primary task of identifying nodules, radiographers in this condition also had to interpret where another person looked in relation to each image and determine whether this information was useful in helping them make their own decisions. In this way, there were multiple cognitive demands associated with interpreting another person’s eye movements that would have been likely to require longer decision times. Intriguingly, Experiment 1 revealed no difference between those radiographers watching an expert search for nodules and those radiographers watching a novice search for nodules. Although Nalanagula et al. (2006) have shown that novices improve when shown an expert’s eye movement patterns in an eye movement feed-forward paradigm, the present findings indicate that this improvement can also be obtained from viewing a novice’s eye movement patterns. There is an implicit assumption that improved performance will be confined to the eye movement patterns of experts, because an expert should be more likely to lead observers to the most task-relevant areas (e.g., Kundel & Nodine, 1983; Nodine et al., 2002). However, it is not the case that novices totally ignore task-relevant areas. On the contrary, Table 2 shows that the distribution of fixations at nodule locations was similar for both the expert and novice models (collapsed across images). Therefore, the novice’s eye movements may have also led observers to consider these areas. Providing that the novice was looking at a nodule, even if the novice did not make the correct decision or was less confident than the expert, the novice’s eye movements could have still been useful in guiding observers to consider these areas. << Insert Table 2 about here >> In line with this latter argument it seems that the primary factor in learning from another’s gaze relates to the task-specificity of the eye movements, rather than the Viewing Another Person’s Eye Movements 16 expertise of the individual who provided them (see Land, 2006; Yarbus, 1967, for evidence supporting the task-specific nature of eye movements). Indeed, we suggest that if an observer viewed eye movements that were unrelated to the nodule-detection task then there should be no performance benefit because these eye movements would not guide attention towards task-relevant areas. This proposal, however, requires empirical validation, as it is also possible that the performance benefit observed in the eye movement preview condition arose simply because this condition elevated the general attention or arousal of participants, thereby leading to more aggressive scanning of images. Experiment 2 directly addressed these issues by comparing the effects of viewing an expert’s task-specific eye movement patterns versus a set of eye movement patterns that were unrelated to the nodule-detection task. A final issue to contend with is how the radiographers perceived the model. In Experiment 1 the identity and level of expertise of the model were kept ambiguous. Research has shown that integrating advice from others into the decision-making process can involve some degree of authority bias and is influenced by the observer’s own level of experience, and the experience of the advisors (Harvey & Fischer, 1997; Schrah, Dalal, & Sniezek, 2006). Although it is conceivable that these adjustments may occur in the present study, we would still argue that the task-specificity of the eye movement behavior would remain as the primary factor in learning from another’s eye movements. That is, explicitly knowing that the search behavior came from an expert might well influence the uptake of potentially helpful information, but in the case of unhelpful information (e.g., in the form of unrelated eye movement patterns) such knowledge should be of little value. Experiment 2 Experiment 2 aimed to establish whether the performance improvement in Experiment 1 was caused by the presentation of task-specific eye movement patterns or Viewing Another Person’s Eye Movements 17 whether it was a consequence of providing dynamic visual stimuli during image preview that led to a general increase in attention or arousal. To arbitrate between these alternative accounts Experiment 2 involved three conditions: (1) image-preview control, which was not associated with any eye movement cues; (2) “expert-search” preview, which used the expert eye movement patterns from Experiment 1 as visual cues; and (3) “unrelated” preview, which used eye-movement patterns related to a different visual task as visual cues (i.e., an observer’s free search of features within fractal images). In line with previous evidence for the task-specific nature of eye-movements (Land, 2006; Yarbus, 1967) we predicted that performance improvements in Experiment 2 would arise only in the expert-search preview condition, even though participants in both eye movement conditions were explicitly told that they would be following an expert radiologist. Method Participants. Sixty participants were selected for the study: 30 first-year radiographers (8 male; 22 female) and 30 experienced radiographers (10 male; 20 female) with a minimum of five years experience interpreting medical images from a range of modalities. Participants had normal or corrected-to-normal vision and were each paid £6 for completing the experiment. Materials. Forty chest x-rays were selected from the test bank described in Experiment 1. Twenty images were normal and contained no nodules and 20 were abnormal and contained up to 5 nodules in each image, with a total of 46 nodules across the 20 abnormal images. Images were standardized to fit an LCD display (resolution of 1280 x 1024) and were presented in a random order using ClearView 2.6. Design. The experiment employed a 2 x 3 between participants design with two factors: participant expertise, with two levels (novice vs. experienced); and viewing condition, with three levels (image preview; expert-search preview; unrelated preview). Viewing Another Person’s Eye Movements 18 Participants were randomly assigned to one of the three viewing conditions. Image preview was identical to Experiment 1, requiring participants to view each x-ray for 20s before being allowed to mark the location of any nodules. The expert-search preview was the same as the eye movement preview in Experiment 1, with the exception that all the eye movement recordings were from the highest performing expert radiologist (i.e., no novice eye movements were included). To obtain the unrelated preview a nonradiographer was asked to look at 40 fractal images for 20s each. The individual was free to examine the fractals in any manner whilst eye movements were recorded using a Tobii x50 eye tracker. The fractal images varied in complexity and spatial configuration, and it was assumed that the non-radiographer would look at whatever features within each image attracted attention. The resulting dynamic eye movement pattern for each fractal was extracted and superimposed over a randomly selected chest x-ray using AVI video editing software. This produced an identical set of chest x-rays to the expert-search preview condition, with the only difference being that the eye movements in the unrelated preview condition had no direct association to the chest x-ray image (i.e., it was purely by chance whether the gaze cursor landed on a nodule). Table 3 shows the distribution of fixations for the unrelated and expert-search eye movements at nodule locations. It is evident that all metrics for the expert-search eyemovements show higher values at nodule locations than for the unrelated eye movements. These data attest to the task-relevance of the eye movement patterns. It is also apparent that at least some fixations fall at nodule locations in the unrelated preview condition. << Insert Table 3 about here >> Procedure. Participants had to identify nodules following the same procedure as Experiment 1. Participants in the image preview condition viewed each image for 20s before making a decision. Participants in the unrelated and the expert-search preview Viewing Another Person’s Eye Movements 19 conditions were told they would be “shown an eye movement recording indicating where an expert radiologist examined each image.” As such, participants in the unrelated preview condition believed they were following an expert’s eye movement patterns. As with Experiment 1, participants in the unrelated and the expert-search preview conditions were not told whether the expert’s diagnosis for each image had been correct. Results and Discussion The figure-of-merit scores obtained from the JAFROC analysis showed a main effect of viewing condition [F(2, 54) = 8.53, p < .001, partial η = .24], with performance significantly better with expert-search preview compared to image preview and unrelated preview (see Table 4). Performance between novice and experienced radiographers was significantly different, with experienced radiographers outperforming novices [F(1, 54) = 12.00, p < .001, partial η = .18]. A reliable interaction was also found between participant expertise and viewing condition [F(2, 54) = 4.07, p < .023, partial η = .13]. << Insert Table 4 about here >> Simple main effects analyses were undertaken for each level of expertise. Novice radiographers showed a reliable performance difference across conditions [F(2, 27) = 12.53, p < .001, partial η = .48]. Bonferroni-corrected pairwise comparisons confirmed that the expert-search preview condition led to higher performance than the image preview and unrelated preview conditions (both ps < .001). The unrelated preview and image-preview conditions did not differ (p > .20). Experienced radiographers did not show a reliable difference in performance across conditions [F(2, 27) = 0.47, p > .20, partial η = .03]. These analyses indicate that only novice radiographers benefited from viewing an expert’s search related eye movement patterns before making a decision regarding nodules. Simple main effects analyses were undertaken for each level of viewing condition. Experienced radiographers were significantly better than novice Viewing Another Person’s Eye Movements 20 radiographers in the image preview condition [F(1, 19) = 7.15, p < .016] and the unrelated preview condition [F(1, 19) = 11.84, p < .003] but not in the expert-search preview condition [F(1, 19) = 0.13, p > .20]. Overall, these analyses clarify that the novice and experienced radiographers only performed equivalently when eye movement cues were task related. Furthermore, these findings support the claim that viewing another’s search behavior aids performance by guiding observers’ towards task-relevant areas. The alternative hypothesis – that performance benefits arise simply because eye movement cues elevate participants’ general attention or arousal – received no support. All decision times for the three viewing conditions in Experiment 2 included 20s of preview time (see Table 4). The distribution of decision times indicated that one novice radiographer was over 2.5 SD from the mean; this participant’s data were removed from the analysis. There was no main effect of viewing condition [F(2, 53) = 1.67, p > .20, partial η = .06], however, there was a main effect of expertise [F(1, 53) = 4.04, p < .049, partial η = .07), with novice radiographers taking longer than experienced radiographers. No interaction was found between expertise and viewing condition [F(2, 53) = .81, p > .20, partial η = .03]. These results indicate that the experienced radiographers were faster than novices in diagnosing each image by 7.67s on average. Viewing task-related eye movement patterns once again facilitated performance, but only for novice radiographers. The fact that this improvement was not consistently found by experienced radiographers suggests this benefit may be short-lived and restricted to the early stages of learning. In contrast to Experiment 1, following and interpreting task-specific eye movements was not accompanied with increased decision times relative to the image preview condition. This lack of a difference may reflect the fact that the number of trials per condition increased from 14 in Experiment 1 to 40 in Experiment 2. As a result, participants in Experiment 2 had more time to become familiar Viewing Another Person’s Eye Movements 21 with interpreting the model’s eye movements and to learn how to integrate another’s search behavior with their own decision-making. The latter methodological difference may also explain why experience-related differences in JAFROC scores were more noticeable, despite the smaller number of participants per condition in Experiment 2. A key finding from Experiment 2 was that viewing unrelated eye movement patterns did not lead to performance benefits, despite participants being told they would be following where a consultant radiologist looked. Hence the improvement in the expertsearch preview condition was not simply due to a general increase in attention or to the observer’s knowledge that an expert was helping them. Instead, following task-specific eye movements must have provided some additional information as to how to interpret each x-ray. Understanding what separates the eye movement patterns of the expert search and unrelated conditions should reveal what this additional information might be. Table 3 shows that the expert-search model looked at nodules more often and for longer, whereas in the unrelated preview condition, gaze position coincided with nodules less frequently and for shorter duration. As such, it is likely that observers in the expertsearch preview condition were guided towards nodules more often than observers in the unrelated preview condition and that it is this increase in attentional guidance towards task-relevant areas that explains the performance benefit. Similarly, where the eye movement metrics indicated no difference in attentional guidance, for example, between the expert and novice model in Experiment 1, we found no difference in performance.